Statistical Methods for Clinical Study Site Selection.

BACKGROUND: The US Food and Drug Administration conducts on-site inspections and data audits through Bioresearch Monitoring program for assurance of the quality and integrity of data in the pre- and postapproval processes. It is important to inspect the study sites that are different compared with other sites in clinical studies and identify the problems related to those sites. Usually one cannot inspect all the sites in a clinical study because of limited resources, and statistical tools are needed to help in selecting sites for inspection. METHODS: We propose two technical approaches, namely Fisher combination approach and likelihood ratio test (LRT) approach, for site selection, with each approach integrating the information obtained from a P value matrix. The proposed approaches produce site rankings, and the sites with highest rankings may be selected for inspection. RESULTS: The application of the approaches is demonstrated through a hypothetical data set reflecting the pattern of the real data in a premarket approval submission for a diagnostic device. The proposed methods are shown, through extensive simulations, to control false discovery rate, while maintaining good sensitivity. CONCLUSION: The proposed approaches will be useful for site selection process. However, limitations exist when only using the statistical approaches proposed here. In practice, investigators will select the site for inspection by considering the outputs from the statistical approaches along with other important factors. Future research topic is discussed to facilitate practical application of the approaches.

Likelihood-Ratio-Test Methods for Drug Safety Signal Detection from Multiple Clinical Datasets.

Pre- and postmarket drug safety evaluations usually include an integrated summary of results obtained using data from multiple studies related to a drug of interest. This paper proposes three approaches based on the likelihood ratio test (LRT), called the LRT methods, for drug safety signal detection from large observational databases with multiple studies, with focus on identifying signals of adverse events (AEs) from many AEs associated with a particular drug or inversely for signals of drugs associated with a particular AE. The methods discussed include simple pooled LRT method and its variations such as the weighted LRT that incorporates the total drug exposure information by study. The power and type-I error of the LRT methods are evaluated in a simulation study with varying heterogeneity across studies. For illustration purpose, these methods are applied to Proton Pump Inhibitors (PPIs) data with 6 studies for the effect of concomitant use of PPIs in treating patients with osteoporosis and to Lipiodol (a contrast agent) data with 13 studies for evaluating that drug's safety profiles.

Hierarchical Bayes approach for subgroup analysis.

In clinical data analysis, both treatment effect estimation and consistency assessment are important for a better understanding of the drug efficacy for the benefit of subjects in individual subgroups. The linear mixed-effects model has been used for subgroup analysis to describe treatment differences among subgroups with great flexibility. The hierarchical Bayes approach has been applied to linear mixed-effects model to derive the posterior distributions of overall and subgroup treatment effects. In this article, we discuss the prior selection for variance components in hierarchical Bayes, estimation and decision making of the overall treatment effect, as well as consistency assessment of the treatment effects across the subgroups based on the posterior predictive p-value. Decision procedures are suggested using either the posterior probability or the Bayes factor. These decision procedures and their properties are illustrated using a simulated example with normally distributed response and repeated measurements.

Zero-inflated Poisson model based likelihood ratio test for drug safety signal detection.

In recent decades, numerous methods have been developed for data mining of large drug safety databases, such as Food and Drug Administration's (FDA's) Adverse Event Reporting System, where data matrices are formed by drugs such as columns and adverse events as rows. Often, a large number of cells in these data matrices have zero cell counts and some of them are "true zeros" indicating that the drug-adverse event pairs cannot occur, and these zero counts are distinguished from the other zero counts that are modeled zero counts and simply indicate that the drug-adverse event pairs have not occurred yet or have not been reported yet. In this paper, a zero-inflated Poisson model based likelihood ratio test method is proposed to identify drug-adverse event pairs that have disproportionately high reporting rates, which are also called signals. The maximum likelihood estimates of the model parameters of zero-inflated Poisson model based likelihood ratio test are obtained using the expectation and maximization algorithm. The zero-inflated Poisson model based likelihood ratio test is also modified to handle the stratified analyses for binary and categorical covariates (e.g. gender and age) in the data. The proposed zero-inflated Poisson model based likelihood ratio test method is shown to asymptotically control the type I error and false discovery rate, and its finite sample performance for signal detection is evaluated through a simulation study. The simulation results show that the zero-inflated Poisson model based likelihood ratio test method performs similar to Poisson model based likelihood ratio test method when the estimated percentage of true zeros in the database is small. Both the zero-inflated Poisson model based likelihood ratio test and likelihood ratio test methods are applied to six selected drugs, from the 2006 to 2011 Adverse Event Reporting System database, with varying percentages of observed zero-count cells.

Likelihood ratio based tests for longitudinal drug safety data.

This article presents longitudinal likelihood ratio test (LongLRT) methods for large databases with exposure information. These methods are applied to a pooled large longitudinal clinical trial dataset for drugs treating osteoporosis with concomitant use of proton pump inhibitors (PPIs). When the interest is in the evaluation of a signal of an adverse event for a particular drug compared with placebo or a comparator, the special case of the LongLRT, referred to as sequential LRT (SeqLRT), is also presented. The results show that there is some possible evidence of concomitant use of PPIs leading to more adverse events associated with osteoporosis. The performance of the proposed LongLRT and SeqLRT methods is evaluated using simulated datasets and shown to be good in terms of (conditional) power and control of type I error over time. The proposed methods can also be applied to large observational databases with exposure information under the US Food and Drug Administration Sentinel Initiative for active surveillance. Published 2014. This article is a US Government work and is in the public domain in the USA.

A Review of Statistical Methods for Safety Surveillance.

The data-mining statistical methods used for disproportionality analysis of drug-adverse event combinations from large drug safety databases such as the FDA's Adverse Event Reporting System (FAERS), consisting of spontaneous reports on adverse events for postmarket drugs, are called passive surveillance methods. However, the statistical signal detection methods for longitudinal data, as the data accrue in time, are called active surveillance methods. A review of the most commonly used passive surveillance statistical methods and the relationships among them is presented with unified notations. These methods are applied to the 2006-2012 FAERS data; the number of drug signals of disproportionate rates (SDRs) detected by each of these methods with the common SDRs from all of these methods, for the adverse event myocardial infarction, are given. Finally, there is a brief discussion on the recently developed active surveillance methods.

Likelihood ratio test-based method for signal detection in drug classes using FDA's AERS database.

In 1968 the Food and Drug Administration (FDA) established the Adverse Event Reporting System (AERS) database containing data on adverse events (AEs) reported by patients, health care providers, and other sources through a spontaneous reporting system. FDA uses AERS for monitoring the safety of the drugs on the market after approval. Most statistical methods that are available in the literature to analyze large postmarket drug safety data for identifying drug-event combinations with disproportionately high frequencies are designed to explore signals of a single drug-AE combination, but not signals including a drug class or a group of AEs simultaneously. Those methods are also not designed to control type I error and are subject to high false discovery rates. In this paper, we first briefly review a recently developed method, known as the likelihood ratio test (LRT)-based method, which has been demonstrated to control the family-wise type I error and false discovery rates. By introducing a concept of weight matrix for the drugs (or for AEs), we then extend the LRT method for detecting signals including a class of drugs (or AEs) in addition to detecting signals of single drug (or AE). A simplified Bayesian method is also proposed and compared with LRT method. The proposed methods are applied to study the signal patterns of drug classes, namely, the gadolinium drug class for magnetic resonance imaging (MRI) and statins for hypercholesterolemia, over different time periods, using the datasets with only suspect drugs and with both suspect and concomitant drugs from the AERS database. The signals detected by the statistical methods can be confirmed by signals detected across different databases, existing medical evidence from research or regulatory resources, prospective biological studies, and also through simulation as illustrated in the application.